<!DOCTYPE html>
<!--

	Modified template for STM32CubeMX.AI purpose

	d0.1: 	jean-michel.delorme@st.com
			add ST logo and ST footer

	d2.0: 	jean-michel.delorme@st.com
			add sidenav support

	d2.1: 	jean-michel.delorme@st.com
			clean-up + optional ai_logo/ai meta data
			
==============================================================================
           "GitHub HTML5 Pandoc Template" v2.1 — by Tristano Ajmone           
==============================================================================
Copyright © Tristano Ajmone, 2017, MIT License (MIT). Project's home:

- https://github.com/tajmone/pandoc-goodies

The CSS in this template reuses source code taken from the following projects:

- GitHub Markdown CSS: Copyright © Sindre Sorhus, MIT License (MIT):
  https://github.com/sindresorhus/github-markdown-css

- Primer CSS: Copyright © 2016-2017 GitHub Inc., MIT License (MIT):
  http://primercss.io/

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The MIT License 

Copyright (c) Tristano Ajmone, 2017 (github.com/tajmone/pandoc-goodies)
Copyright (c) Sindre Sorhus <sindresorhus@gmail.com> (sindresorhus.com)
Copyright (c) 2017 GitHub Inc.

"GitHub Pandoc HTML5 Template" is Copyright (c) Tristano Ajmone, 2017, released
under the MIT License (MIT); it contains readaptations of substantial portions
of the following third party softwares:

(1) "GitHub Markdown CSS", Copyright (c) Sindre Sorhus, MIT License (MIT).
(2) "Primer CSS", Copyright (c) 2016 GitHub Inc., MIT License (MIT).

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
==============================================================================-->
<html>
<head>
  <meta charset="utf-8" />
  <meta name="generator" content="pandoc" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
  <meta name="keywords" content="STM32CubeMX, X-CUBE-AI, Neural Network, Quantization, CLI, Code Generator, Automatic NN mapping tools" />
  <title>Quantization and quantize command</title>
  <style type="text/css">
.markdown-body{
	-ms-text-size-adjust:100%;
	-webkit-text-size-adjust:100%;
	color:#24292e;
	font-family:-apple-system,system-ui,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol";
	font-size:16px;
	line-height:1.5;
	word-wrap:break-word;
	box-sizing:border-box;
	min-width:200px;
	max-width:980px;
	margin:0 auto;
	padding:45px;
	}
.markdown-body a{
	color:#0366d6;
	background-color:transparent;
	text-decoration:none;
	-webkit-text-decoration-skip:objects}
.markdown-body a:active,.markdown-body a:hover{
	outline-width:0}
.markdown-body a:hover{
	text-decoration:underline}
.markdown-body a:not([href]){
	color:inherit;text-decoration:none}
.markdown-body strong{font-weight:600}
.markdown-body h1,.markdown-body h2,.markdown-body h3,.markdown-body h4,.markdown-body h5,.markdown-body h6{
	margin-top:24px;
	margin-bottom:16px;
	font-weight:600;
	line-height:1.25}
.markdown-body h1{
	font-size:2em;
	margin:.67em 0;
	padding-bottom:.3em;
	border-bottom:1px solid #eaecef}
.markdown-body h2{
	padding-bottom:.3em;
	font-size:1.5em;
	border-bottom:1px solid #eaecef}
.markdown-body h3{font-size:1.25em}
.markdown-body h4{font-size:1em}
.markdown-body h5{font-size:.875em}
.markdown-body h6{font-size:.85em;color:#6a737d}
.markdown-body img{border-style:none}
.markdown-body svg:not(:root){
	overflow:hidden}
.markdown-body hr{
	box-sizing:content-box;
	height:.25em;
	margin:24px 0;
	padding:0;
	overflow:hidden;
	background-color:#e1e4e8;
	border:0}
.markdown-body hr::before{display:table;content:""}
.markdown-body hr::after{display:table;clear:both;content:""}
.markdown-body input{margin:0;overflow:visible;font:inherit;font-family:inherit;font-size:inherit;line-height:inherit}
.markdown-body [type=checkbox]{box-sizing:border-box;padding:0}
.markdown-body *{box-sizing:border-box}.markdown-body blockquote{margin:0}
.markdown-body ol,.markdown-body ul{padding-left:2em}
.markdown-body ol ol,.markdown-body ul ol{list-style-type:lower-roman}
.markdown-body ol ol,.markdown-body ol ul,.markdown-body ul ol,.markdown-body ul ul{margin-top:0;margin-bottom:0}
.markdown-body ol ol ol,.markdown-body ol ul ol,.markdown-body ul ol ol,.markdown-body ul ul ol{list-style-type:lower-alpha}
.markdown-body li>p{margin-top:16px}
.markdown-body li+li{margin-top:.25em}
.markdown-body dd{margin-left:0}
.markdown-body dl{padding:0}
.markdown-body dl dt{padding:0;margin-top:16px;font-size:1em;font-style:italic;font-weight:600}
.markdown-body dl dd{padding:0 16px;margin-bottom:16px}
.markdown-body code{font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,Courier,monospace}
.markdown-body pre{font:12px SFMono-Regular,Consolas,"Liberation Mono",Menlo,Courier,monospace;word-wrap:normal}
.markdown-body blockquote,.markdown-body dl,.markdown-body ol,.markdown-body p,.markdown-body pre,.markdown-body table,.markdown-body ul{margin-top:0;margin-bottom:16px}
.markdown-body blockquote{padding:0 1em;color:#6a737d;border-left:.25em solid #dfe2e5}
.markdown-body blockquote>:first-child{margin-top:0}
.markdown-body blockquote>:last-child{margin-bottom:0}
.markdown-body table{display:block;width:100%;overflow:auto;border-spacing:0;border-collapse:collapse}
.markdown-body table th{font-weight:600}
.markdown-body table td,.markdown-body table th{padding:6px 13px;border:1px solid #dfe2e5}
.markdown-body table tr{background-color:#fff;border-top:1px solid #c6cbd1}
.markdown-body table tr:nth-child(2n){background-color:#f6f8fa}
.markdown-body img{max-width:100%;box-sizing:content-box;background-color:#fff}
.markdown-body code{padding:.2em 0;margin:0;font-size:85%;background-color:rgba(27,31,35,.05);border-radius:3px}
.markdown-body code::after,.markdown-body code::before{letter-spacing:-.2em;content:"\00a0"}
.markdown-body pre>code{padding:0;margin:0;font-size:100%;word-break:normal;white-space:pre;background:0 0;border:0}
.markdown-body .highlight{margin-bottom:16px}
.markdown-body .highlight pre{margin-bottom:0;word-break:normal}
.markdown-body .highlight pre,.markdown-body pre{padding:16px;overflow:auto;font-size:85%;line-height:1.45;background-color:#f6f8fa;border-radius:3px}
.markdown-body pre code{display:inline;max-width:auto;padding:0;margin:0;overflow:visible;line-height:inherit;word-wrap:normal;background-color:transparent;border:0}
.markdown-body pre code::after,.markdown-body pre code::before{content:normal}
.markdown-body .full-commit .btn-outline:not(:disabled):hover{color:#005cc5;border-color:#005cc5}
.markdown-body kbd{box-shadow:inset 0 -1px 0 #959da5;display:inline-block;padding:3px 5px;font:11px/10px SFMono-Regular,Consolas,"Liberation Mono",Menlo,Courier,monospace;color:#444d56;vertical-align:middle;background-color:#fcfcfc;border:1px solid #c6cbd1;border-bottom-color:#959da5;border-radius:3px;box-shadow:inset 0 -1px 0 #959da5}
.markdown-body :checked+.radio-label{position:relative;z-index:1;border-color:#0366d6}
.markdown-body .task-list-item{list-style-type:none}
.markdown-body .task-list-item+.task-list-item{margin-top:3px}
.markdown-body .task-list-item input{margin:0 .2em .25em -1.6em;vertical-align:middle}
.markdown-body::before{display:table;content:""}
.markdown-body::after{display:table;clear:both;content:""}
.markdown-body>:first-child{margin-top:0!important}
.markdown-body>:last-child{margin-bottom:0!important}
.Alert,.Error,.Note,.Success,.Warning{padding:11px;margin-bottom:24px;border-style:solid;border-width:1px;border-radius:4px}
.Alert p,.Error p,.Note p,.Success p,.Warning p{margin-top:0}
.Alert p:last-child,.Error p:last-child,.Note p:last-child,.Success p:last-child,.Warning p:last-child{margin-bottom:0}
.Alert{color:#246;background-color:#e2eef9;border-color:#bac6d3}
.Warning{color:#4c4a42;background-color:#fff9ea;border-color:#dfd8c2}
.Error{color:#911;background-color:#fcdede;border-color:#d2b2b2}
.Success{color:#22662c;background-color:#e2f9e5;border-color:#bad3be}
.Note{color:#2f363d;background-color:#f6f8fa;border-color:#d5d8da}
.Alert h1,.Alert h2,.Alert h3,.Alert h4,.Alert h5,.Alert h6{color:#246;margin-bottom:0}
.Warning h1,.Warning h2,.Warning h3,.Warning h4,.Warning h5,.Warning h6{color:#4c4a42;margin-bottom:0}
.Error h1,.Error h2,.Error h3,.Error h4,.Error h5,.Error h6{color:#911;margin-bottom:0}
.Success h1,.Success h2,.Success h3,.Success h4,.Success h5,.Success h6{color:#22662c;margin-bottom:0}
.Note h1,.Note h2,.Note h3,.Note h4,.Note h5,.Note h6{color:#2f363d;margin-bottom:0}
.Alert h1:first-child,.Alert h2:first-child,.Alert h3:first-child,.Alert h4:first-child,.Alert h5:first-child,.Alert h6:first-child,.Error h1:first-child,.Error h2:first-child,.Error h3:first-child,.Error h4:first-child,.Error h5:first-child,.Error h6:first-child,.Note h1:first-child,.Note h2:first-child,.Note h3:first-child,.Note h4:first-child,.Note h5:first-child,.Note h6:first-child,.Success h1:first-child,.Success h2:first-child,.Success h3:first-child,.Success h4:first-child,.Success h5:first-child,.Success h6:first-child,.Warning h1:first-child,.Warning h2:first-child,.Warning h3:first-child,.Warning h4:first-child,.Warning h5:first-child,.Warning h6:first-child{margin-top:0}
h1.title,p.subtitle{text-align:center}
h1.title.followed-by-subtitle{margin-bottom:0}
p.subtitle{font-size:1.5em;font-weight:600;line-height:1.25;margin-top:0;margin-bottom:16px;padding-bottom:.3em}
div.line-block{white-space:pre-line}
  </style>
  <style type="text/css">code{white-space: pre;}</style>
  <style type="text/css">
	code.sourceCode > span { display: inline-block; line-height: 1.25; }
code.sourceCode > span { color: inherit; text-decoration: inherit; }
code.sourceCode > span:empty { height: 1.2em; }
.sourceCode { overflow: visible; }
code.sourceCode { white-space: pre; position: relative; }
div.sourceCode { margin: 1em 0; }
pre.sourceCode { margin: 0; }
@media screen {
div.sourceCode { overflow: auto; }
}
@media print {
code.sourceCode { white-space: pre-wrap; }
code.sourceCode > span { text-indent: -5em; padding-left: 5em; }
}
pre.numberSource code
  { counter-reset: source-line 0; }
pre.numberSource code > span
  { position: relative; left: -4em; counter-increment: source-line; }
pre.numberSource code > span > a:first-child::before
  { content: counter(source-line);
    position: relative; left: -1em; text-align: right; vertical-align: baseline;
    border: none; display: inline-block;
    -webkit-touch-callout: none; -webkit-user-select: none;
    -khtml-user-select: none; -moz-user-select: none;
    -ms-user-select: none; user-select: none;
    padding: 0 4px; width: 4em;
    color: #aaaaaa;
  }
pre.numberSource { margin-left: 3em; border-left: 1px solid #aaaaaa;  padding-left: 4px; }
div.sourceCode
  {   }
@media screen {
code.sourceCode > span > a:first-child::before { text-decoration: underline; }
}
code span.al { color: #ff0000; font-weight: bold; } /* Alert */
code span.an { color: #60a0b0; font-weight: bold; font-style: italic; } /* Annotation */
code span.at { color: #7d9029; } /* Attribute */
code span.bn { color: #40a070; } /* BaseN */
code span.bu { } /* BuiltIn */
code span.cf { color: #007020; font-weight: bold; } /* ControlFlow */
code span.ch { color: #4070a0; } /* Char */
code span.cn { color: #880000; } /* Constant */
code span.co { color: #60a0b0; font-style: italic; } /* Comment */
code span.cv { color: #60a0b0; font-weight: bold; font-style: italic; } /* CommentVar */
code span.do { color: #ba2121; font-style: italic; } /* Documentation */
code span.dt { color: #902000; } /* DataType */
code span.dv { color: #40a070; } /* DecVal */
code span.er { color: #ff0000; font-weight: bold; } /* Error */
code span.ex { } /* Extension */
code span.fl { color: #40a070; } /* Float */
code span.fu { color: #06287e; } /* Function */
code span.im { } /* Import */
code span.in { color: #60a0b0; font-weight: bold; font-style: italic; } /* Information */
code span.kw { color: #007020; font-weight: bold; } /* Keyword */
code span.op { color: #666666; } /* Operator */
code span.ot { color: #007020; } /* Other */
code span.pp { color: #bc7a00; } /* Preprocessor */
code span.sc { color: #4070a0; } /* SpecialChar */
code span.ss { color: #bb6688; } /* SpecialString */
code span.st { color: #4070a0; } /* String */
code span.va { color: #19177c; } /* Variable */
code span.vs { color: #4070a0; } /* VerbatimString */
code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warning */
  </style>
  <style type="text/css">:root { --main-hx-color: rgb(0,32,88); --sidenav-font-size: 90%;}html {}* {xbox-sizing: border-box;}.st_header h1.title,.st_header p.subtitle {text-align: left;}.st_header h1.title {color: var(--main-hx-color)}.st_header p.subtitle {color: var(--main-hx-color)}.st_header h1.title.followed-by-subtitle {margin-bottom:5px;}.st_header p.revision {display: inline-block;width:70%;}.st_header div.author {font-style: italic;}.st_header div.summary {border-top: solid 1px #C0C0C0;background: #ECECEC;padding: 5px;}.st_footer img {float: right;}.markdown-body #header-section-number {font-size:120%;}.markdown-body h1 {border-bottom:1px solid #74767a;padding-bottom: 2px;padding-top: 10px;}.markdown-body h2 {padding-bottom: 5px;padding-top: 10px;}.markdown-body h2 code {background-color: rgb(255, 255, 255);}#func.sourceCode {border-left-style: solid;border-color: rgb(0, 32, 82);border-color: rgb(255, 244, 191);border-width: 8px;padding:0px;}pre > code {border: solid 1px blue;font-size:60%;}codeXX {border: solid 1px blue;font-size:60%;}#func.sourceXXCode::before {content: "Synopsis";padding-left:10px;font-weight: bold;}figure {padding:0px;margin-left:5px;margin-right:5px;margin-left: auto;margin-right: auto;}img[data-property="center"] {display: block;margin-top: 10px;margin-left: auto;margin-right: auto;padding: 10px;}figcaption {text-align:left;  border-top: 1px dotted #888;padding-bottom: 20px;margin-top: 10px;}section.st_footer {font-size:80%;}div.stnotice {width:80%;}h1 code, h2 code {font-size:120%;}	.markdown-body table {width: 100%;margin-left:auto;margin-right:auto;}.markdown-body img {border-radius: 4px;padding: 5px;display: block;margin-left: auto;margin-right: auto;width: auto;}.markdown-body .st_header img, .markdown-body {border: none;border-radius: none;padding: 5px;display: block;margin-left: auto;margin-right: auto;width: auto;box-shadow: none;}.markdown-body {margin: 10px;padding: 10px;width: auto;font-family: "Arial", sans-serif;color: #03234B;}.markdown-body h1, .markdown-body h2, .markdown-body h3 {   color: var(--main-hx-color)}.markdown-body:hover {}.markdown-body .contents {}.markdown-body .toc-title {}.markdown-body .contents li {list-style-type: none;}.markdown-body .contents ul {padding-left: 10px;}.markdown-body .contents a {color: #3CB4E6; }.sidenav {font-family: "Arial", sans-serif;font-family: segoe ui, verdona;color: #3CB4E6; color: #03234B; color: var(--main-hx-color);height: 100%;position: fixed;z-index: 1;top: 0;left: 0;margin-right: 10px;margin-left: 10px; overflow-x: hidden;}hr.new1 {border-width: thin;border-top: 1px solid #3CB4E6; margin-right: 10px;margin-top: -10px;}.sidenav #sidenav_header {margin-top: 10px;border: 1px;}.sidenav #sidenav_header img {float: left;}.sidenav #sidenav_header a {margin-left: 0px;margin-right: 0px;padding-left: 0px;color: #3CB4E6; color: #03234B; color: var(--main-hx-color)}.sidenav #sidenav_header a:hover {background-size: auto;color: #FFD200; }.sidenav #sidenav_header a:active {  }.sidenav > ul {background-color: rgba(57, 169, 220, 0.05);border-radius: 10px;padding-bottom: 10px;padding-top: 10px;padding-right: 10px;margin-right: 10px;}.sidenav a {padding: 2px 2px;text-decoration: none;font-size: var(--sidenav-font-size);  display:table;}.sidenav > ul > li,.sidenav > ul > li > ul > li { padding-right: 5px;padding-left: 5px;}.sidenav > ul > li > a { color: #03234B;  color: var(--main-hx-color)}.sidenav > ul > li > ul > li > a { color: #03234B; color: #3CB4E6; color: #03234B; font-weight: lighter;padding-left: 10px;}.sidenav > ul > li > ul > li > ul > li > a { display: None;}.sidenav li {list-style-type: none;}.sidenav ul {padding-left: 0px;}.sidenav > ul > li > a:hover,.sidenav > ul > li > ul > li > a:hover {background-color: rgba(70, 70, 80, 0.1); background-clip: border-box;margin-left: -10px;padding-left: 10px;}.sidenav > ul > li > a:hover {padding-right: 15px;width: 230px;	}.sidenav > ul > li > ul > li > a:hover {padding-right: 10px;width: 230px;	}.sidenav > ul > li > a:active { color: #FFD200; }.sidenav code {}.sidenav {width: 280px;}#sidenav {margin-left: 300px;display:block;}.markdown-body .print-contents {visibility:hidden;}.markdown-body .print-toc-title {visibility:hidden;}.markdown-body {max-width: 980px;min-width: 200px;padding: 40px;border-style: solid;border-style: outset;border-color: rgba(104, 167, 238, 0.089);border-radius: 5px;}@media screen and (max-height: 450px) {.sidenav {padding-top: 15px;}.sidenav a {font-size: 18px;}#sidenav {margin-left: 10px; }.sidenav {visibility:hidden;}.markdown-body {margin: 10px;padding: 40px;width: auto;border: 0px;}}@media screen and (max-width: 1024px) {.sidenav {visibility:hidden;}.markdown-body {margin: 10px;padding: 40px;width: auto;border: 0px;}#sidenav {margin-left: 10px;}}@media print {.sidenav {visibility:hidden;}#sidenav {margin-left: 10px;}.markdown-body {margin: 10px;padding: 10px;width:auto;border: 0px;}@page {size: A4;  margin:2cm;padding:2cm;margin-top: 1cm;padding-bottom: 1cm;}* {xbox-sizing: border-box;font-size:90%;}a {font-size: 100%;color: yellow;}.markdown-body article {xbox-sizing: border-box;font-size:100%;}.markdown-body p {windows: 2;orphans: 2;}.pagebreakerafter {page-break-after: always;padding-top:10mm;}.pagebreakbefore {page-break-before: always;}h1, h2, h3, h4 {page-break-after: avoid;}div, code, blockquote, li, span, table, figure {page-break-inside: avoid;}}</style>
  <!--[if lt IE 9]>
    <script src="//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv-printshiv.min.js"></script>
  <![endif]-->




<link href="" rel="shortcut icon">

</head>



<body>

		<div class="sidenav">
		<div id="sidenav_header">
							<img src="" title="STM32CubeMX.AI logo" align="left" height="70" />
										<br />5.2.0<br />
										<a href="#doc_title"> Quantization and quantize command </a>
					</div>
		<div id="sidenav_header_button">
			 
							<ul>
					<li><p><a id="index" href="index.html">[ Index ]</a></p></li>
				</ul>
						<hr class="new1">
		</div>	

		<ul>
<li><a href="#introduction">Introduction</a><ul>
<li><a href="#quantization-support">Quantization support</a></li>
<li><a href="#ref_support_arithmetic">Supported 8b integer format</a></li>
<li><a href="#ref_tf_support">TensorFlow lite support</a></li>
</ul></li>
<li><a href="#ref_quantize_cmd">Quantize command</a><ul>
<li><a href="#overview">Overview</a></li>
<li><a href="#examples">Examples</a></li>
<li><a href="#ref_quant_conf_file">Post-training quantization configuration file</a></li>
<li><a href="#ref_quant_flow">Keras Post-training quantization process</a></li>
<li><a href="#ref_supported_layers">Supported Keras layers</a></li>
<li><a href="#ref_test_sets_loading">Test-set considerations</a></li>
<li><a href="#ref_quant_algo">Quantizers</a></li>
<li><a href="#ref_tensor_conf_file">Tensor format configuration file</a></li>
<li><a href="#ref_quant_mnist">Quantize a MNIST model</a></li>
</ul></li>
<li><a href="#references">References</a></li>
<li><a href="#revision-history">Revision history</a></li>
</ul>
	</div>
	<article id="sidenav" class="markdown-body">
	


<header>
<section class="st_header" id="doc_title">

<div class="himage">
	<img src="" title="STM32CubeMX.AI" align="right" height="70" />
	<img src="" title="STM32" align="right" height="90" />
</div>

<h1 class="title followed-by-subtitle">Quantization and quantize command</h1>

	<p class="subtitle">X-CUBE-AI Expansion Package</p>

	<div class="revision">r2.1</div>

	<div class="ai_platform">
		AI PLATFORM r5.2.0
					(Embedded Inference Client API 1.1.0)
			</div>
			Command Line Interface r1.4.0
	




</section>
</header>




<section id="introduction" class="level1">
<h1>Introduction</h1>
<section id="quantization-support" class="level2">
<h2>Quantization support</h2>
<p>X-CUBE-AI code generator can be used to generate and to deploy a quantized model (<a href="#ref_support_arithmetic">8b integer format</a>). Quantization (also called calibration) is an optimization technique to deploy a 32b float model reducing the size (smaller storage size and less memory peak usage at runtime) and improving CPU/MCU and hardware accelerator latency (including power consumption aspects) with little degradation in model accuracy. It is an important part of various optimization techniques: topology-oriented, features-map reduction, pruning, weights compression… which can be applied to address the resource-constrained environment.</p>
<p>There are two forms of quantization: post-training quantization and quantization aware training. First is easier to use, it allows to quantize a pre-trained model with a limited and representative data set. Quantization aware training is done during the training process and is often better for model accuracy.</p>
<figure>
<img src="" property="center" style="width:95.0%" alt />
</figure>
<p>The CLI provides an internal post-training quantization process (see <a href="#ref_quantize_cmd">“quantize” command</a> section) with different <a href="#ref_support_arithmetic">quantization schemes</a> for an already-trained Keras model.</p>
<p>X-CUBE-AI is able to import two sort of quantized model:</p>
<ul>
<li>a Keras float model associated with its <a href="#ref_tensor_conf_file">tensor format configuration</a> file. The conversion of each 32b float weight/bias tensors to 8b integer format is directly achieved by the importer engine thanks to the provided settings. No calibration is performed.</li>
<li>a quantized TensorFlow lite model (generated by a post-training or training aware process). In this case, the conversion has been performed by the TensorFlow Lite framework, principally through the “TFLite converter” utility exporting the <a href="#ref_tf_support">TensorFlow lite</a> file. No calibration is performed.</li>
</ul>
<p>For a given layer/operator, weights and activations should be quantized. Full 8b integer format is requested, weights only or float16 TFLite quantization variants are not supported. The mixed models with the convert operators (like QUANTIZE or DEQUANTIZE Tensor Lite operators), explicitly defined or automatically inserted by the X-CUBE-AI code generator are supported. Finally the quantized tensors are mapped on the optimized and specialized C-implementation for the supported operators (see <a href="layer-support.html">[3]</a>), otherwise the floating-point version of the operator is used.</p>
<div class="Warning">
<p><strong>Note</strong> — As the Keras API is now fully integrated in the TensorFlow framework, the user has the possibility to quantize a Keras model with the TFLite Converter utility (post-training quantization) and/or the X-CUBE-AI <a href="#ref_quantize_cmd">internal process</a>. Despite the current limitations, this internal process offers more <a href="#ref_support_arithmetic">quantization schemes</a> which can be more interesting in term of execution time and precision (i.e. accuracy). Results are fully dependent of the model size and associated topology. It provides also currently a better support to deploy a model with the recurrent layers which are kept in float w/o extra manipulation.</p>
</div>
<p>The “analyze”, “validate” and “generate” commands can be used w/o limitations. The <code>&#39;-q/--quantize&#39;</code> argument is used to pass the specific “tensor format configuration” file for a given Keras model.</p>
<pre class="dosbatch"><code>$ stm32ai analyze -m &lt;reshaped_model_file&gt;.h5 -q &lt;quant_file_desc&gt;.json
$ stm32ai analyze -m &lt;quantized_model_file&gt;.tflite
$ stm32ai validate -m &lt;quantized_model&gt;.tflite -vi test_data.npz</code></pre>
<div class="Note">
<p><strong>Note</strong> — By default, if a Softmax layer is part of the network, it is automatically keep in float.</p>
</div>
</section>
<section id="ref_support_arithmetic" class="level2">
<h2>Supported 8b integer format</h2>
<p>X-CUBE-AI supports two 8b integer-base arithmetics: <strong>Qm,n</strong> and <strong>Integer</strong>.</p>
<p>Legacy <strong>Qm,n</strong> arithmetic (introduced in X-CUBE-AI 4.0) is a signed fixed-point number format (two’s complement) where the number of fractional bits <em>n</em> and the number of integer bits <em>m</em> are specified with a constant and fixed resolution. For example, Q0.7 number has 7 fractional bits; Q1.6 number has 1 integer bit and 6 fractional bits. By convention <em>m</em> does not include the sign bit.</p>
<figure>
<img src="" property="center" style="width:60.0%" alt />
</figure>
<div class="Error">
<p><strong>Note</strong> – <em>Qm,n</em> arithmetic is a legacy support, it will be deprecated in future release.</p>
</div>
<p><strong>Integer</strong> arithmetic (introduced in X-CUBE-AI 4.1) is based on a more representative convention used by Google for the quantized models. See the following reference to highlight the underlying rational.</p>
<ul>
<li>Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference (<a href="https://arxiv.org/abs/1712.05877">https://arxiv.org/abs/1712.05877</a>)</li>
</ul>
<p>For the integer arithmetic, each real number <em>r</em> is represented in function of the quantized value <em>q</em>, a <em>scale</em> factor (<em>arbitrary positive real number</em>) and a <em>zero_point</em> parameter. Quantization scheme is an affine mapping of the integers <em>q</em> to real numbers <em>r</em>.</p>
<figure>
<img src="" property="center" style="width:60.0%" alt />
</figure>
<blockquote>
<p><em>zero_point</em> has the same integer C-type like the <em>q</em> data.</p>
</blockquote>
<p>For <strong>Qm,n</strong>, fixed point format with power of 2 scaling, centered to zero, the precision is directly deduced and fixed by the <em>m,n</em> parameters. For the <strong>Integer</strong> arithmetic, precision is dependent of a <em>scale</em> factor and the quantized values are linearly distributed around the <em>zero_point</em> value. In both case, resolution/precision is constant vs floating-point representation.</p>
<div id="fig:id_quant" class="fignos">
<figure>
<img src="" property="center" style="width:95.0%" alt /><figcaption><span>Figure 1:</span> Integer vs Qm,n precision</figcaption>
</figure>
</div>
<section id="per-axis-or-per-channel-vs-per-tensor" class="level3">
<h3>Per-axis (or per-channel) vs per-tensor</h3>
<p>Currently <strong>per-tensor</strong> quantization is supported for <em>Qm,n</em> and <em>Integer</em> arithmetics. This means that the same format is used for the entire tensor. For <em>Integer</em> arithemtic <strong>only</strong>, <strong>per-axis</strong> (or <strong>per-channel</strong>) for conv-base operator is supported, this means there will be one <em>scale</em> and/or <em>zero_point</em> per slice.</p>
<p>Activation tensors are always in <strong>per-tensor</strong>.</p>
</section>
<section id="symmetric-vs-asymmetric" class="level3">
<h3>Symmetric vs Asymmetric</h3>
<p><strong>Asymmetric</strong> means that the tensor can have <em>zero_point</em> anywhere within the signed 8b range [-128, 127] or unsigned 8b range [0, 255]. <strong>Symmetric</strong> means that the tensor is forced to have <em>zero_point</em> equal to zero. By enforcing <em>zero_point</em> to zero, some kernel optimization implementations are possible to limit the cost of the operations (off-line pre-calculation,…). By nature, the activations are asymmetric, consequently symmetric format for the activations is not supported. For the weights/bias, asymmetric and symmetric format are supported.</p>
<p>For <em>Qm,n</em> arithmetic, only symmetric format is considered.</p>
</section>
<section id="signed-integer-vs-unsigned-integer---supported-schemes" class="level3">
<h3>Signed integer vs Unsigned integer - supported schemes</h3>
<p>Signed or unsigned integer type can be defined for the weights and/or activations. However all requested kernels are not implemented or relevant to support the different optimized combinations related to the symmetric and asymmetric format. This imply that <em>only</em> the following integer schemes or combinations are supported:</p>
<table>
<colgroup>
<col style="width: 28%"></col>
<col style="width: 36%"></col>
<col style="width: 36%"></col>
</colgroup>
<thead>
<tr class="header">
<th style="text-align: left;">scheme</th>
<th style="text-align: left;">weights</th>
<th style="text-align: left;">activations</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td style="text-align: left;">ua/ua</td>
<td style="text-align: left;">unsigned and asymmetric</td>
<td style="text-align: left;">unsigned and asymmetric</td>
</tr>
<tr class="even">
<td style="text-align: left;">ss/sa</td>
<td style="text-align: left;">signed and symmetric</td>
<td style="text-align: left;">signed and asymmetric</td>
</tr>
<tr class="odd">
<td style="text-align: left;">ss/ua</td>
<td style="text-align: left;">signed and symmetric</td>
<td style="text-align: left;">unsigned and asymmetric</td>
</tr>
</tbody>
</table>
<p>For <em>Qm,n</em> arithmetic, only equivalent “ss/ss” scheme is considered. Implementation of the kernels are specific.</p>
</section>
</section>
<section id="ref_tf_support" class="level2">
<h2>TensorFlow lite support</h2>
<p>X-CUBE-AI is able to import the training-aware and post-training quantized TensorFlow lite models. Post-training quantized models (TensorFlow v1.15 or v2.x) are based on the “ss/sa” and per-channel scheme. Activations are asymmetric and signed (int8), weights/bias are symmetric and signed (int8). Previous training-aware quantized models are based on the “ua/ua” scheme. Now the “ss/sa” and per-channel scheme is also the privileged scheme to address efficiently the <a href="https://coral.ai/docs/edgetpu/models-intro/#compatibility-overview">Coral Edge TPUs</a> or <a href="https://www.tensorflow.org/lite/microcontrollers">TensorFlow Lite for Microcontrollers</a> runtime.</p>
<p>For X-CUBE-AI, following code snippet illustrates the requested <em>TFLiteConverter</em> options to enforce full integer post-training quantization for all operators including the input/output tensors.</p>
<div class="Note">
<p><strong>Note</strong> — The quantization of the input or/and output tensors are optional. They can be conserved in float for convenience and deployment facility, for example to keep the pre or/and post-processes in float.</p>
</div>
<div class="sourceCode" id="cb2"><pre class="sourceCode python"><code class="sourceCode python"><span id="cb2-1"><a href="#cb2-1"></a><span class="kw">def</span> representative_dataset_gen():</span>
<span id="cb2-2"><a href="#cb2-2"></a>  data <span class="op">=</span> tload(...)</span>
<span id="cb2-3"><a href="#cb2-3"></a></span>
<span id="cb2-4"><a href="#cb2-4"></a>  <span class="cf">for</span> _ <span class="kw">in</span> <span class="bu">range</span>(num_calibration_steps):</span>
<span id="cb2-5"><a href="#cb2-5"></a>    <span class="co"># Get sample input data as a numpy array in a method of your choosing.</span></span>
<span id="cb2-6"><a href="#cb2-6"></a>    <span class="bu">input</span> <span class="op">=</span> get_sample(data)</span>
<span id="cb2-7"><a href="#cb2-7"></a>    <span class="cf">yield</span> [<span class="bu">input</span>]</span>
<span id="cb2-8"><a href="#cb2-8"></a></span>
<span id="cb2-9"><a href="#cb2-9"></a>converter <span class="op">=</span> tf.lite.TFLiteConverter.from_keras_model_file(<span class="op">&lt;</span>keras_model_path<span class="op">&gt;</span>)</span>
<span id="cb2-10"><a href="#cb2-10"></a><span class="co"># trick from 2.x environment</span></span>
<span id="cb2-11"><a href="#cb2-11"></a><span class="co"># converter = tf.compat.v1.lite.TFLiteConverter.from_keras_model_file(&lt;keras_model_path&gt;)</span></span>
<span id="cb2-12"><a href="#cb2-12"></a>converter.representative_dataset <span class="op">=</span> representative_dataset_gen</span>
<span id="cb2-13"><a href="#cb2-13"></a><span class="co"># This enables quantization</span></span>
<span id="cb2-14"><a href="#cb2-14"></a>converter.optimizations <span class="op">=</span> [tf.lite.Optimize.DEFAULT]</span>
<span id="cb2-15"><a href="#cb2-15"></a><span class="co"># This ensures that if any ops can&#39;t be quantized, the converter throws an error</span></span>
<span id="cb2-16"><a href="#cb2-16"></a>converter.target_spec.supported_ops <span class="op">=</span> [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]</span>
<span id="cb2-17"><a href="#cb2-17"></a><span class="co"># For full integer quantization, though supported types defaults to int8 only</span></span>
<span id="cb2-18"><a href="#cb2-18"></a>converter.target_spec.supported_types <span class="op">=</span> [tf.int8]</span>
<span id="cb2-19"><a href="#cb2-19"></a><span class="co"># These set the input and output tensors to uint8 (added in r2.3)</span></span>
<span id="cb2-20"><a href="#cb2-20"></a>converter.inference_input_type <span class="op">=</span> tf.uint8  <span class="co"># or tf.int8</span></span>
<span id="cb2-21"><a href="#cb2-21"></a>converter.inference_output_type <span class="op">=</span> tf.uint8  <span class="co"># or tf.int8</span></span>
<span id="cb2-22"><a href="#cb2-22"></a>quant_model <span class="op">=</span> converter.convert()</span>
<span id="cb2-23"><a href="#cb2-23"></a></span>
<span id="cb2-24"><a href="#cb2-24"></a><span class="co"># Save t quatized file</span></span>
<span id="cb2-25"><a href="#cb2-25"></a><span class="cf">with</span> <span class="bu">open</span>(<span class="op">&lt;</span>tflite_quant_model_path<span class="op">&gt;</span>, <span class="st">&quot;wb&quot;</span>) <span class="im">as</span> f:</span>
<span id="cb2-26"><a href="#cb2-26"></a>    f.write(quant_model)</span>
<span id="cb2-27"><a href="#cb2-27"></a>...</span></code></pre></div>
<ul>
<li>Post-training quantization: <a href="https://www.tensorflow.org/lite/performance/post_training_quantization">https://www.tensorflow.org/lite/performance/post_training_quantization</a><br />
</li>
<li>Quantization aware training: <a href="https://www.tensorflow.org/model_optimization/guide/quantization/training">https://www.tensorflow.org/model_optimization/guide/quantization/training</a></li>
</ul>
<div class="Note">
<p><strong>Note</strong> — TensorFlow Lite framework is used to deploy a deep learning model on mobile and embedded devices. Generated TFLite file is a self-contained file containing a frozen description of the graph, setting of the operators and the tensors (including the data). 32b float and quantized models are supported. This file is directly used by a runtime interpreter, see <a href="https://www.tensorflow.org/lite/microcontrollers">TensorFlow Lite for Microcontrollers</a>, or as entry point for a compiler like <a href="https://coral.ai/docs/edgetpu/models-intro/#compatibility-overview">Coral Edge TPUs compiler</a> or code generator like X-CUBE-AI to create an adapted and optimized version targeting a particular hardware: MPU, MCU or hardware assist IP. Functional point of view, content of the file is similar to the generate <code>&lt;network&gt;.c</code> and <code>&lt;network&gt;_data.c</code> files, implementation of the kernels should be provided by the runtime.</p>
</div>
</section>
</section>
<section id="ref_quantize_cmd" class="level1">
<h1>Quantize command</h1>
<section id="overview" class="level2">
<h2>Overview</h2>
<p>The “quantize” command allows to perform a <a href="#ref_quant_flow">post-training quantization process</a> on a 32b float Keras model allowing to generate a new Keras model (reshaped model) file and associated <a href="#ref_tensor_conf_file">tensor format configuration</a> file compatible with the X-CUBE-AI code generator. Options are passed through a <a href="#ref_quant_conf_file">post-training quantization configuration</a> json file (<code>&#39;-q/--quantize&#39;</code> argument). An additional JSON file is generated because Keras h5 format does not provide natively a support to handle quantized params or meta information. Note that the reshaped model file is basically an un-fused version of the original 32b float model which can be used as-is.</p>
<figure>
<img src="" property="center" style="width:95.0%" alt />
</figure>
<p>Post-training quantization process limitations</p>
<ul>
<li>residual or multi-branches model</li>
<li>model with multiple inputs or outputs</li>
<li>only channel last (NHWC) tensor representation is supported</li>
<li>only the supported floating-point Keras operators (refer to <a href="layer-support.html">[4]</a>) can be quantized. Layers that are not supported for <a href="#ref_supported_layers">fixed-point conversion</a> will be kept in floating point.</li>
</ul>
</section>
<section id="examples" class="level2">
<h2>Examples</h2>
<ul>
<li><p>Perform the post-training quantization process on an already-trained floating-point Keras model.</p>
<pre class="dosbatch"><code>$ stm32ai quantize -q &lt;conf_quant&gt;.json</code></pre></li>
<li><p>Validate a Keras model after Keras post-training quantization</p>
<pre class="dosbatch"><code>$ stm32ai validate -m &lt;reshaped_model_file&gt;.h5 -q &lt;quant_file_desc&gt;.json -vi test_data.npz</code></pre></li>
<li><p>Generate the specialized files for a quantized Keras model.</p>
<pre class="dosbatch"><code>$ stm32ai generate -m &lt;expanded_model_file&gt;.h5 -q &lt;quant_file_desc&gt;.json</code></pre></li>
</ul>
</section>
<section id="ref_quant_conf_file" class="level2">
<h2>Post-training quantization configuration file</h2>
<p>The use the Keras post-training quantization process a configuration file (JSON dictionary) is requested with the following keys:</p>
<table>
<colgroup>
<col style="width: 32%"></col>
<col style="width: 67%"></col>
</colgroup>
<thead>
<tr class="header">
<th style="text-align: left;">Key</th>
<th style="text-align: left;">Description</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td style="text-align: left;">“model_name”</td>
<td style="text-align: left;">indicates the name/suffix used for the produced files.<br />
- <code>&lt;model_name&gt;.h5</code> - reshaped model file<br />
- <code>&lt;model_name&gt;_Q.json</code> - tensor format configuration file<br />
- <code>&lt;model_name&gt;_reference.npz</code> - reference file</td>
</tr>
<tr class="even">
<td style="text-align: left;">“path_to_floatingpoint_h5”</td>
<td style="text-align: left;">indicates the path to the original model file.</td>
</tr>
<tr class="odd">
<td style="text-align: left;">“algorithm”</td>
<td style="text-align: left;">indicates the used algorithm. Possible values: <em>“User”</em>, <em>“Greedy”</em> or <em>“Minmax”</em> (see <em><a href="#ref_quant_algo">“Quantizers”</a></em> section)</td>
</tr>
<tr class="even">
<td style="text-align: left;">“arithmetic”</td>
<td style="text-align: left;">indicates the expected arithmetic. Possible values: <em>“Integer”</em>, <em>“Qmn”</em> (see <em><a href="#ref_support_arithmetic">“Supported integer format”</a></em> section)</td>
</tr>
<tr class="odd">
<td style="text-align: left;">“weights_integer_scheme”</td>
<td style="text-align: left;">indicates the expected scheme for the weights (<em>“Integer”</em> arithmetic only). Possible values: <em>“UnsignedAsymmetric”</em>, <em>“SignedSymmetric”</em> (see <em><a href="#ref_support_arithmetic">“Supported 8b integer format”</a></em> section)</td>
</tr>
<tr class="even">
<td style="text-align: left;">“activations_integer_scheme”</td>
<td style="text-align: left;">indicates the expected scheme for the activations (<em>“Integer”</em> arithmetic only). Possible values: <em>“UnsignedAsymmetric”</em>, <em>“SignedAsymmetric”</em> (see <em><a href="#ref_support_arithmetic">“Supported 8b integer format”</a></em> section)</td>
</tr>
<tr class="odd">
<td style="text-align: left;">“per_channel”</td>
<td style="text-align: left;">indicates if <em>per-channel</em> (or <em>per-axis</em>) quantization sub-mode must be applied for the weights/bias. Activation tensors are still in per-tensor mode. This option is only applicable for the “integer” arithmetic. Possible values: “True” or “False”. If not defined, “False” is considered and <em>per-tensor</em> (or <em>per-layer</em>) quantization sub-mode is used (see <em><a href="#ref_support_arithmetic">“Supported integer format”</a></em> section).</td>
</tr>
<tr class="even">
<td style="text-align: left;">“quant_test_set_dir”</td>
<td style="text-align: left;">indicates the <em>quantization test-set</em> directory (see <em><a href="#ref_test_sets_loading">“Test-set considerations”</a></em> section).</td>
</tr>
<tr class="odd">
<td style="text-align: left;">“evaluation_test_set_dir”</td>
<td style="text-align: left;">indicates the <em>evaluation test-set</em> directory (see <em><a href="#ref_test_sets_loading">“Test-set considerations”</a></em> section).</td>
</tr>
<tr class="even">
<td style="text-align: left;">“batch_size”</td>
<td style="text-align: left;">indicates number of inputs vectors processed for one <em>evaluation</em>. The user needs to choose carefully this parameter in function of its system memory. Recommendation is to start with a small value for example 32.</td>
</tr>
<tr class="odd">
<td style="text-align: left;">“quant_test_ratio”</td>
<td style="text-align: left;">indicates the ratio <code>[0..1]</code> of the vectors in “quant_test_set_dir” which are used for the quantization. They are randomly selected.</td>
</tr>
<tr class="even">
<td style="text-align: left;">“output_directory”</td>
<td style="text-align: left;">indicates the root directory to store the results. Produced files are stored in the following directory: <code>&lt;output_directory&gt;/&lt;model_name&gt;_&lt;algorithm&gt;_&lt;date&gt;_&lt;time&gt;/</code>.</td>
</tr>
<tr class="odd">
<td style="text-align: left;">“modules_directory”</td>
<td style="text-align: left;">indicates the directory containing the user test-set generation and optional user quantizer Python files.</td>
</tr>
<tr class="even">
<td style="text-align: left;">“filename_test_set_generation”</td>
<td style="text-align: left;">name of the file (with or without <code>py</code> extension), where the user writes a potential pre-processing of the data, and in any case, load the test-sets into generators. This file is <strong>mandatory</strong>.</td>
</tr>
<tr class="odd">
<td style="text-align: left;">“filename_quantizer_algos”</td>
<td style="text-align: left;">name of the file (with or without <code>py</code> extension), where the user can write his own quantizer. This is only needed if <em>“User”</em> algorithm is requested.</td>
</tr>
</tbody>
</table>
<div class="Note">
<p><strong>Note</strong> – For the “path_to_floatingpoint_h5”, “output_directory”, “modules_directory”, “quant_test_set_dir” and “evaluation_test_set_dir” keys if the values are not prefixed by <code>&quot;./&quot;</code> or <code>&quot;/&quot;</code>, path is relative to the location of the JSON file path. Otherwise, the path is absolute or relative to the current executing path where the stm32ai application is launched.</p>
</div>
<div class="Warning">
<p><strong>Note</strong> – All fields are requested. Only the <em>“filename_quantizer_algos”</em> can be omitted if <em>“User”</em> algo is not selected.</p>
</div>
<p><strong>“quant_test_ratio” parameter</strong></p>
<p>If you have <em>1000</em> images in “quant_test_set_dir” and the user sets “quant_test_ratio” to <em>0.8</em> then only <em>800</em> images we be used from the quantization test-set. This parameter can be viewed as a way to control the execution time of the script that can be long depending on the user system, the size of the quantization test-set, the quantization algorithm and the depth of the network. Please, note that in case Keras pre-existing <em>ImageDataGenerator()</em> class is used to load data, then Keras constraints impose <em>“quant_test_ratio”</em> be strictly lower than <em>1</em>.</p>
<section id="examples-of-configuration-files" class="level3 unnumbered">
<h3>Examples of configuration files</h3>
<section id="x-cube-ai-4.0-legacy-config---qmn-arithmetic-and-greedy-algo" class="level4 unnumbered">
<h4>X-CUBE-AI 4.0 legacy config - “Qmn” arithmetic and “Greedy” algo</h4>
<div class="Warning">
<p><strong>Note</strong> — <em>“weights_integer_scheme”</em> and <em>“activations_integer_scheme”</em> entries should be defined but they are not taken into account.</p>
</div>
<div class="sourceCode" id="cb6"><pre class="sourceCode json"><code class="sourceCode json"><span id="cb6-1"><a href="#cb6-1"></a><span class="fu">{</span></span>
<span id="cb6-2"><a href="#cb6-2"></a>        <span class="dt">&quot;model_name&quot;</span><span class="fu">:</span> <span class="st">&quot;mnist&quot;</span><span class="fu">,</span></span>
<span id="cb6-3"><a href="#cb6-3"></a>        <span class="dt">&quot;path_to_floatingpoint_h5&quot;</span><span class="fu">:</span> <span class="st">&quot;mnist_cnn.h5&quot;</span><span class="fu">,</span></span>
<span id="cb6-4"><a href="#cb6-4"></a>        <span class="dt">&quot;batch_size&quot;</span><span class="fu">:</span> <span class="st">&quot;128&quot;</span><span class="fu">,</span></span>
<span id="cb6-5"><a href="#cb6-5"></a>        <span class="dt">&quot;quant_test_set_dir&quot;</span><span class="fu">:</span> <span class="st">&quot;.</span><span class="ch">\\</span><span class="st">&quot;</span><span class="fu">,</span></span>
<span id="cb6-6"><a href="#cb6-6"></a>        <span class="dt">&quot;quant_test_ratio&quot;</span><span class="fu">:</span> <span class="st">&quot;0.3&quot;</span><span class="fu">,</span></span>
<span id="cb6-7"><a href="#cb6-7"></a>        <span class="dt">&quot;evaluation_test_set_dir&quot;</span><span class="fu">:</span> <span class="st">&quot;.</span><span class="ch">\\</span><span class="st">&quot;</span><span class="fu">,</span></span>
<span id="cb6-8"><a href="#cb6-8"></a>        <span class="dt">&quot;modules_directory&quot;</span><span class="fu">:</span> <span class="st">&quot;.</span><span class="ch">\\</span><span class="st">mnist_modules</span><span class="ch">\\</span><span class="st">&quot;</span><span class="fu">,</span></span>
<span id="cb6-9"><a href="#cb6-9"></a>        <span class="dt">&quot;filename_test_set_generation&quot;</span><span class="fu">:</span> <span class="st">&quot;test_set_generation&quot;</span><span class="fu">,</span></span>
<span id="cb6-10"><a href="#cb6-10"></a>        <span class="dt">&quot;filename_quantizer_algos&quot;</span><span class="fu">:</span> <span class="st">&quot;quantizer_user_algo&quot;</span><span class="fu">,</span></span>
<span id="cb6-11"><a href="#cb6-11"></a>        <span class="dt">&quot;algorithm&quot;</span><span class="fu">:</span> <span class="st">&quot;Greedy&quot;</span><span class="fu">,</span></span>
<span id="cb6-12"><a href="#cb6-12"></a>        <span class="dt">&quot;arithmetic&quot;</span><span class="fu">:</span> <span class="st">&quot;Qmn&quot;</span><span class="fu">,</span></span>
<span id="cb6-13"><a href="#cb6-13"></a>        <span class="dt">&quot;weights_integer_scheme&quot;</span><span class="fu">:</span> <span class="st">&quot;UnsignedAsymmetric&quot;</span><span class="fu">,</span></span>
<span id="cb6-14"><a href="#cb6-14"></a>        <span class="dt">&quot;activations_integer_scheme&quot;</span><span class="fu">:</span> <span class="st">&quot;UnsignedAsymmetric&quot;</span><span class="fu">,</span></span>
<span id="cb6-15"><a href="#cb6-15"></a>        <span class="dt">&quot;output_directory&quot;</span><span class="fu">:</span> <span class="st">&quot;out&quot;</span></span>
<span id="cb6-16"><a href="#cb6-16"></a><span class="fu">}</span></span></code></pre></div>
</section>
<section id="integer-arithmetic-with-uaua-scheme" class="level4">
<h4>“Integer” arithmetic with Ua/Ua scheme</h4>
<div class="sourceCode" id="cb7"><pre class="sourceCode json"><code class="sourceCode json"><span id="cb7-1"><a href="#cb7-1"></a><span class="fu">{</span></span>
<span id="cb7-2"><a href="#cb7-2"></a>        <span class="dt">&quot;model_name&quot;</span><span class="fu">:</span> <span class="st">&quot;mnist&quot;</span><span class="fu">,</span></span>
<span id="cb7-3"><a href="#cb7-3"></a>        <span class="dt">&quot;path_to_floatingpoint_h5&quot;</span><span class="fu">:</span> <span class="st">&quot;mnist_cnn.h5&quot;</span><span class="fu">,</span></span>
<span id="cb7-4"><a href="#cb7-4"></a>        <span class="dt">&quot;batch_size&quot;</span><span class="fu">:</span> <span class="st">&quot;128&quot;</span><span class="fu">,</span></span>
<span id="cb7-5"><a href="#cb7-5"></a>        <span class="dt">&quot;quant_test_set_dir&quot;</span><span class="fu">:</span> <span class="st">&quot;.</span><span class="ch">\\</span><span class="st">&quot;</span><span class="fu">,</span></span>
<span id="cb7-6"><a href="#cb7-6"></a>        <span class="dt">&quot;quant_test_ratio&quot;</span><span class="fu">:</span> <span class="st">&quot;0.3&quot;</span><span class="fu">,</span></span>
<span id="cb7-7"><a href="#cb7-7"></a>        <span class="dt">&quot;evaluation_test_set_dir&quot;</span><span class="fu">:</span> <span class="st">&quot;.</span><span class="ch">\\</span><span class="st">&quot;</span><span class="fu">,</span></span>
<span id="cb7-8"><a href="#cb7-8"></a>        <span class="dt">&quot;modules_directory&quot;</span><span class="fu">:</span> <span class="st">&quot;.</span><span class="ch">\\</span><span class="st">mnist_modules</span><span class="ch">\\</span><span class="st">&quot;</span><span class="fu">,</span></span>
<span id="cb7-9"><a href="#cb7-9"></a>        <span class="dt">&quot;filename_test_set_generation&quot;</span><span class="fu">:</span> <span class="st">&quot;test_set_generation&quot;</span><span class="fu">,</span></span>
<span id="cb7-10"><a href="#cb7-10"></a>        <span class="dt">&quot;filename_quantizer_algos&quot;</span><span class="fu">:</span> <span class="st">&quot;quantizer_user_algo&quot;</span><span class="fu">,</span></span>
<span id="cb7-11"><a href="#cb7-11"></a>        <span class="dt">&quot;algorithm&quot;</span><span class="fu">:</span> <span class="st">&quot;MinMax&quot;</span><span class="fu">,</span></span>
<span id="cb7-12"><a href="#cb7-12"></a>        <span class="dt">&quot;arithmetic&quot;</span><span class="fu">:</span> <span class="st">&quot;Integer&quot;</span><span class="fu">,</span></span>
<span id="cb7-13"><a href="#cb7-13"></a>        <span class="dt">&quot;weights_integer_scheme&quot;</span><span class="fu">:</span> <span class="st">&quot;UnsignedAsymmetric&quot;</span><span class="fu">,</span></span>
<span id="cb7-14"><a href="#cb7-14"></a>        <span class="dt">&quot;activations_integer_scheme&quot;</span><span class="fu">:</span> <span class="st">&quot;UnsignedAsymmetric&quot;</span><span class="fu">,</span></span>
<span id="cb7-15"><a href="#cb7-15"></a>        <span class="dt">&quot;output_directory&quot;</span><span class="fu">:</span> <span class="st">&quot;out&quot;</span></span>
<span id="cb7-16"><a href="#cb7-16"></a><span class="fu">}</span></span></code></pre></div>
</section>
</section>
</section>
<section id="ref_quant_flow" class="level2">
<h2>Keras Post-training quantization process</h2>
<p>The Keras post-training quantization process goes through the following steps. For all selected algo or quantized scheme, the same flow is applied.</p>
<div class="Note">
<p><strong>Warning</strong> — Internally the algorithm is fully based on the tf.keras module from the TensorFlow v2.0. It allows to import the h5 file generated with the original Keras module v2.0 up to v2.3.1 and also with the tf.keras from TensorFlow v1.15. Consequently, it is recommended to use also the services from the tf.keras module to design the user modules (<code>test_set_generation.py</code> and <code>quantizer_user_algo.py</code> modules) avoiding possible incompatible situation.</p>
</div>
<div id="fig:id_quant_steps" class="fignos">
<figure>
<img src="" property="center" style="width:65.0%" alt /><figcaption><span>Figure 1:</span> Quantization steps</figcaption>
</figure>
</div>
<p><strong>[1.0]</strong> - Load the <em>evaluation test-set</em>, <em>quantization test-set</em> and original model (see <em><a href="#ref_test_sets_loading">“Test-set considerations”</a></em> section). Note that the Keras <em>load_model()</em> v2.2.4 function is used to load the original model.</p>
<pre class="dosbatch"><code>Neural Network Tools for STM32 v1.1.0 (AI tools v4.1.0)
Post-training quantization v2.0.0
Output directory     : &lt;output_directory&gt;\&lt;model_name&gt;_&lt;algorithm&gt;_&lt;date&gt;_&lt;time&gt;
Requested arithmetic : Qmn
Quantization algo    : greedy
Original model       : &lt;path_to_floatingpoint_h5&gt;
Module directory     : &lt;modules_directory&gt;
-- Loading dataset
USER messages, print(&quot;Load my data set...&quot;)
...
-- Loading dataset - done (elapsed time 0.089s)</code></pre>
<p><strong>[1.1]</strong> - Accuracy of the original model is evaluated with the <em>evaluation test-set</em>. The Keras <em>mean_squared_error</em> function is used as loss function. If no ground truth or reference values are provided, this step is skipped.</p>
<pre class="dosbatch"><code>-- Testing original model
Original model test accuracy (loss): 0.9943 (0.0204)
-- Testing original model - done (elapsed time 0.678s)</code></pre>
<p><strong>[1.2]</strong> - Automatic <strong>reshape</strong> of the original model (see <a href="#ref_supported_layers"><em>“Supported Keras layers”</em></a> section).</p>
<ul>
<li>split a SeparableConv2D into a DepthwiseConv2D followed by a Pointwise Conv2D.</li>
<li>un-fuse activations whenever they are merged into a trainable layer in the original floating-point model</li>
<li>folding of Batch normalization weights (if there is no non-linearity between Batch-Norm and previous trainable layer). If folded, the Batch-norm layer no longer appears in the reshaped model. If Batch-norm cannot be folded, it will be automatically kept in floating point and following message is displayed:</li>
</ul>
<pre class="dosbatch"><code>...
Batch normalisation layer #9 of output model was not folded into previous layer weights
Reason: layer #8 &#39;Activation&#39; is not supported for possible BatchNormalization folding
    supported layers: (&#39;Dense&#39;, &#39;Conv2D&#39;, &#39;DepthwiseConv2D&#39;, &#39;SeparableConv2D&#39;, &#39;Conv1D&#39;,
        &#39;SeparableConv1D&#39;)
...</code></pre>
<pre class="dosbatch"><code>-- Reshaping original model
try to unroll the SeparableConv2D layers...
unfuse activations...
try to fold the BatchNormalization layers...
-- Reshaping original model - done (elapsed time 1.535s)</code></pre>
<p><strong>[1.3]</strong> - Accuracy of the reshaped model is evaluated with the <em>evaluation test-set</em>. If no ground truth or reference values are provided, this step is skipped.</p>
<div class="Note">
<p><strong>Note</strong> — The modified network is expected to be mathematically equivalent to the original model.</p>
</div>
<pre class="dosbatch"><code>-- Testing reshaped model
Reshaped model test accuracy (loss): 0.9943 (0.0204)
-- Testing reshaped model - done (elapsed time 0.772s)</code></pre>
<p><strong>[1.4]</strong> - Save the reshaped model (<code>h5*</code> file creation)</p>
<pre class="dosbatch"><code>-- Saving re-shaped model
-- Saving re-shaped model - done (elapsed time 0.216s)</code></pre>
<p><strong>[2.0]</strong> - <strong>Quantize the weights</strong>: original weights are quantized, by default with the <em>“minmax”</em> algo. If <em>“User”</em> algorithm is set, the <em>WeightsBiasQuantizerUser()</em> function from the user <em>“filename_quantizer_algos.py”</em> module is imported and used (see <em><a href="#ref_quant_algo">“Quantizers”</a></em>). After this step, a function replaces the original weights of the reshaped model with “fake quantized” weights. Note that no <em>evaluation test-set</em> or <em>quantization test-set</em> or iterative algo are used.</p>
<pre class="dosbatch"><code>-- Quantizing weights with &quot;&lt;algorithm&gt;&quot; algo
-- Quantizing weights with &quot;&lt;algorithm&gt;&quot; algo - done (elapsed time 0.256s)</code></pre>
<p><strong>[2.1]</strong> - <strong>Quantize the activations</strong> by passing the <em>quantization test-set</em>. If <em>“User”</em> algorithm is set, the <em>ActivationsQuantizerUser()</em> function from the user <em>“filename_quantizer_algos.py”</em> module is imported and used (see <em><a href="#ref_quant_algo">“Quantizers”</a></em>))</p>
<p>Following traces shows a “Greedy” algo execution. For the other cases, simple line is displayed.</p>
<pre class="dosbatch"><code>-- Quantizing activations with &quot;greedy&quot; algo
Testing quantization of Input...
[===                                               ]   6.09%             4.60s
Best quantization obtained for Input (accuracy): 0.9875
Testing quantization of layer #1...
[=========                                         ]  19.38%            11.00s
Best quantization obtained for layer #1 (accuracy): 0.9875
Testing quantization of layer #2...
[===============                                   ]  31.45%            18.40s
...
-- Quantizing activations with &quot;greedy&quot; algo - done (elapsed time 82.180s)</code></pre>
<p><strong>[2.2]</strong> - Create and save the tensor format configuration file.</p>
<div class="Error">
<p><strong>Note</strong> — By default, if a Softmax layer is part of the network, it is automatically keep in float.</p>
</div>
<p><strong>[3.0]</strong> - Evaluate the accuracy of the quantized model on the <em>evaluation test-set</em>. If no ground truth or reference values are provided, this step is skipped.</p>
<pre class="dosbatch"><code>-- Testing final quantized model
Final quantized model test accuracy (loss): 0.9929 (0.0232)
-- Testing final quantized model - done (elapsed time 1.421s)</code></pre>
<div class="Note">
<p><strong>Note</strong> — As Keras layers do not support quantized values, the behavior of the quantized layers is emulated by quantizing and then rescaling back to float the tensor values; this process is often called <strong>“fake_quantization”</strong>. The quantization changes significantly the outputs of the network and thus you cannot use the original model to validate its result; however, the script generates reference input and outputs you can use with <strong>X-CUBE-AI</strong>’s validation (see next section).</p>
</div>
<p><strong>[3.1]</strong> - If available, save a batch of inputs (“batch_size”) and the predicted values of the quantized model. It can be used as reference to valid the generated C-model.</p>
<pre class="dosbatch"><code>&lt;output_directory&gt;/&lt;model_name&gt;_&lt;algorithm&gt;_&lt;date&gt;_&lt;time&gt;/&lt;model_name&gt;_reference.npz</code></pre>
<p><strong>[3.2]</strong> - Produced files</p>
<pre class="dosbatch"><code>&lt;output_directory&gt;/&lt;model_name&gt;_&lt;algorithm&gt;_&lt;date&gt;_&lt;time&gt;/final_accuracy.txt
&lt;output_directory&gt;/&lt;model_name&gt;_&lt;algorithm&gt;_&lt;date&gt;_&lt;time&gt;/&lt;model_name&gt;.h5
&lt;output_directory&gt;/&lt;model_name&gt;_&lt;algorithm&gt;_&lt;date&gt;_&lt;time&gt;/&lt;model_name&gt;_Q.json
&lt;output_directory&gt;/&lt;model_name&gt;_&lt;algorithm&gt;_&lt;date&gt;_&lt;time&gt;/&lt;model_name&gt;_reference.npz</code></pre>
<p><strong>[2.2]</strong> - Generate and save the tensor format configuration file.</p>
<section id="validation-on-desktop" class="level4 unnumbered">
<h4>Validation on desktop</h4>
<p>Following command can be used to evaluate the quantized model with the generated x86 C-model (refer to <a href="evaluation_metrics.html">[6]</a> about the evaluated metrics).</p>
<pre class="dosbatch"><code>$ stm32ai validate &lt;model_name&gt;.h5 -q &lt;model_name&gt;_Q.json -vi &lt;model_name&gt;_reference.npz
...
model file         : &lt;model_name&gt;.h5
type               : keras (keras_dump) - tf.keras 2.2.4-tf
c_name             : network
compression        : None
quantize           : &lt;model_name&gt;_Q.json
workspace dir      : &lt;workspace-directory-path&gt;
output dir         : &lt;output-directory-path&gt;
vinput files       : &lt;model_name&gt;_reference.npz
...
input              : quantize_conv2d_1_input [784 items, 784 B, ai_i8, Q0.7, (28, 28, 1)]
inputs (total)     : 784 B
output             : softmax_8 [10 items, 40 B, ai_float, FLOAT32, (10,)]
outputs (total)    : 40 B
params #           : 1,199,882 items (4.58 MiB)
macc               : 12,088,202
weights (ro)       : 1,199,884 B (1171.76 KiB) (-75.00%)
activations (rw)   : 27,776 B (27.12 KiB)
ram (total)        : 28,600 B (27.93 KiB) = 27,776 + 784 + 40

...
Evaluation report (summary)
------------------------------------------------------------
Mode                   acc       rmse      mae
------------------------------------------------------------
x86 C-model #1         100.00%   0.000313  0.000028
original model #1      100.00%   0.001655  0.000174
X-cross #1             100.00%   0.001665  0.000183

L2r error : NOT EVALUATED
...</code></pre>
</section>
</section>
<section id="ref_supported_layers" class="level2">
<h2>Supported Keras layers</h2>
<p>Only the following layers can be quantized by the quantization process:</p>
<ul>
<li>Trainable Layers:
<ul>
<li>Dense, Conv2D, DepthwiseConv2D, DepthwiseConv1D, SeparableConv2D (will be split into DepthwiseConv2D + pointwise Conv2D), Conv1D with the following exceptions:
<ul>
<li>“dilation”: values different from 1 are not supported</li>
</ul></li>
</ul></li>
<li>Pooling layers:
<ul>
<li>MaxPooling1D, MaxPooling2D, AveragePooling1D, AveragePooling2D, GlobalMaxPooling1D, GlobalMaxPooling2D, GlobalAveragePooling1D, GlobalAveragePooling2D</li>
</ul></li>
<li>Activations:
<ul>
<li>Linear, ReLU, Sigmoid, Tanh</li>
<li>Advanced: LeakyReLU, PReLU, ELU, ThresholdedReLU</li>
</ul></li>
<li>Various
<ul>
<li>Input Layer</li>
</ul></li>
</ul>
<p>Permute, Flatten, Reshape, RepeatVector layers are not quantized: they are just re-formatting their input tensor. Dropout and SpatialDropout are skipped. Layers that are not supported for fixed-point conversion will be kept in floating point.</p>
<ul>
<li>Only channel-last (NHWC) tensor representation is supported.</li>
</ul>
</section>
<section id="ref_test_sets_loading" class="level2">
<h2>Test-set considerations</h2>
<p>The quantization process gives the possibility to load batches of test vectors. The quantization requires the knowledge of the range of the values of all the tensors. The weights are constant and thus their range is easy to estimate, whereas the <em>activation’s range</em> depends on the input data. An accurate estimate of the activation range requires a test set representative of real data:</p>
<ul>
<li>it means a balanced test set over all the labels to be classified<br />
</li>
<li>test set should be large enough to reflect how variable can be the network inputs and thus to be as representative as possible</li>
</ul>
<p>We call it <strong>quantization test-set</strong>. If this test-set is too small, not representative enough or even unbalanced, quantization will be performed as usual but there is a risk that the quantized model performance be not satisfying. The other test-set, called <em>evaluation test-set</em>, is used to assess performance of floating point model and quantized model and potentially compare them. The <strong>evaluation test-set</strong> and the <strong>quantization test-set</strong> should be as independent as possible to avoid over-estimation of the quantized model performance.</p>
<p>The input vectors is loaded with the <em>create_test_generator()</em> function which is dynamically imported by the core from the user <em>“filename_test_set_generation.py”</em> module. Preprocessing or adaptation of the loaded data should be done in this function (see <em><a href="#ref_quant_mnist">“Quantize a NMIST model”</a></em> sections).</p>
<div class="sourceCode" id="cb20"><pre class="sourceCode python"><code class="sourceCode python"><span id="cb20-1"><a href="#cb20-1"></a><span class="kw">def</span> create_test_generator(quant_test_set_dir, evaluation_test_set_dir,</span>
<span id="cb20-2"><a href="#cb20-2"></a>                          quant_test_ratio, batch_size)</span></code></pre></div>
<div class="Note">
<p><strong>Note</strong> — If the neural network is not used for a classification task, or if labels are not available for the quantizer then in the <em>create_test_generator()</em> function set <code>quant_test_set_labeled</code> to <code>None</code> and <code>eval_test_set_labeled</code> to <code>None</code>.</p>
</div>
</section>
<section id="ref_quant_algo" class="level2">
<h2>Quantizers</h2>
<p>Two quantizers (also called algorithms) are provided to generate the tensor format configuration file: <strong><em>“Minmax”</em></strong> and <strong><em>“Greedy”</em></strong>. The user (<strong><em>“User”</em></strong> algorithm) has also the possibility to provide its own quantization method with different compromises between quantized data precision and saturation, by implementing the <em>estimate</em> method in the quantizer class and passing it to the functions for weight and activation quantization (see <em>“filename_quantizer_algos.py”</em> user module).</p>
<ul>
<li><p><em>“Minmax”</em> invokes the simple and quick quantization process based on min and max of all the tensors for weights and activations. The <em>quantization test-set</em> is used to estimate the activation ranges. The weights are constant and thus their ranges are directly estimated.</p></li>
<li><p><em>“Greedy”</em> invokes an iterative process which tries to improve the picture for activations only, by searching on a per layer basis among a set of possible quantization formats which is the best in terms of classification accuracy. Note that only the accuracy is considered and not the loss function.</p>
<ul>
<li>consequently, if there is no label in the quantization test-set then <em>“Greedy”</em> will not be executed.</li>
<li>be careful, the execution of this algorithm takes much more time than the simple <em>“Minmax”</em> but can also lead to better performance.</li>
</ul></li>
</ul>
<div class="Warning">
<p><strong>Note</strong> — “Greedy” is not available for the “Integer” arithmetic, just “User” or “Minmax”.</p>
</div>
<div class="Note">
<p><strong>Note</strong> — Whatever the algorithm used, the quantizer script issues the quantized model expected accuracy on the <em>evaluation test-set</em> (if labels are present). This is an interesting indication, that may be used to compare different quantizers but the recommendation is to verify that the quantized model generalize well-enough according to your requirements. This has to be done with real <em>field</em> inputs.</p>
</div>
</section>
<section id="ref_tensor_conf_file" class="level2">
<h2>Tensor format configuration file</h2>
<p>The proprietary <em>tensor format configuration</em> file is a JSON dictionary giving the expected tensor format. One entry is defined for each quantized tensor. If a tensor is omitted, the format is float by default (unless it is inferred). As a result, there is no way to indicate that a tensor format is float. The configuration is provided as a JSON file generated from the network structure and it is specific to a neural network model.</p>
<div class="sourceCode" id="cb21"><pre class="sourceCode json"><code class="sourceCode json"><span id="cb21-1"><a href="#cb21-1"></a><span class="fu">{</span></span>
<span id="cb21-2"><a href="#cb21-2"></a>    <span class="dt">&quot;version&quot;</span><span class="fu">:</span> <span class="st">&quot;2.0&quot;</span><span class="fu">,</span></span>
<span id="cb21-3"><a href="#cb21-3"></a>    <span class="dt">&quot;&lt;layer_type&gt;_&lt;idx&gt;_&lt;tensor_name&gt;&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb21-4"><a href="#cb21-4"></a>        <span class="dt">&quot;format&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb21-5"><a href="#cb21-5"></a>            <span class="dt">&quot;class&quot;</span><span class="fu">:</span> <span class="st">&quot;Integer&quot;</span><span class="fu">,</span></span>
<span id="cb21-6"><a href="#cb21-6"></a>            <span class="dt">&quot;type&quot;</span><span class="fu">:</span> <span class="st">&quot;S8&quot;</span><span class="fu">,</span></span>
<span id="cb21-7"><a href="#cb21-7"></a>            <span class="dt">&quot;params&quot;</span><span class="fu">:</span> <span class="ot">[</span> <span class="ot">[</span> <span class="fl">0.0019106452371559892</span> <span class="ot">],[</span> <span class="dv">132</span> <span class="ot">]</span> <span class="ot">]</span><span class="fu">,</span></span>
<span id="cb21-8"><a href="#cb21-8"></a>        <span class="fu">}</span></span>
<span id="cb21-9"><a href="#cb21-9"></a>    <span class="fu">},</span></span>
<span id="cb21-10"><a href="#cb21-10"></a><span class="fu">}</span></span></code></pre></div>
<table>
<colgroup>
<col style="width: 25%"></col>
<col style="width: 74%"></col>
</colgroup>
<thead>
<tr class="header">
<th style="text-align: left;">field</th>
<th style="text-align: left;">description</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td style="text-align: left;">version</td>
<td style="text-align: left;">version/format of the JSON file</td>
</tr>
<tr class="even">
<td style="text-align: left;">name</td>
<td style="text-align: left;"><code>&lt;layer_type&gt;</code>: name/type of tf.keras layer<br />
<code>&lt;idx&gt;</code>: position of the layer in the Keras network<br />
<code>&lt;tensor_name&gt;</code>: designates the conventional name of the associated tensor: “out”, “weights”, “bias”.</td>
</tr>
<tr class="odd">
<td style="text-align: left;">format.class</td>
<td style="text-align: left;">indicates the arithmetic format: can be “FXP” (for “Qmn”) or “Integer”</td>
</tr>
<tr class="even">
<td style="text-align: left;">format.type</td>
<td style="text-align: left;">indicates the type of data: “U8”, “S8” or “S32” (bias in integer)</td>
</tr>
<tr class="odd">
<td style="text-align: left;">format.params</td>
<td style="text-align: left;">indicates the parameters “FXP”: [number of integer bits (M), number of fractional bits (N)]<br />
“Integer”: [[scale value], [zero_point value]]</td>
</tr>
</tbody>
</table>
<p>In case of per channel, the output JSON file may look like this for example with 8 output channels layer whose weights are quantized in UnsignedAsymmetric:</p>
<div class="sourceCode" id="cb22"><pre class="sourceCode json"><code class="sourceCode json"><span id="cb22-1"><a href="#cb22-1"></a><span class="fu">{</span></span>
<span id="cb22-2"><a href="#cb22-2"></a>    <span class="dt">&quot;version&quot;</span><span class="fu">:</span> <span class="st">&quot;2.0&quot;</span><span class="fu">,</span></span>
<span id="cb22-3"><a href="#cb22-3"></a>    <span class="dt">&quot;&lt;layer_type&gt;_&lt;idx&gt;_&lt;tensor_name&gt;&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb22-4"><a href="#cb22-4"></a>        <span class="dt">&quot;format&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb22-5"><a href="#cb22-5"></a>            <span class="dt">&quot;class&quot;</span><span class="fu">:</span> <span class="st">&quot;Integer&quot;</span><span class="fu">,</span></span>
<span id="cb22-6"><a href="#cb22-6"></a>            <span class="dt">&quot;type&quot;</span><span class="fu">:</span> <span class="st">&quot;U8&quot;</span><span class="fu">,</span></span>
<span id="cb22-7"><a href="#cb22-7"></a>            <span class="dt">&quot;params&quot;</span><span class="fu">:</span> <span class="ot">[</span></span>
<span id="cb22-8"><a href="#cb22-8"></a>                <span class="ot">[</span></span>
<span id="cb22-9"><a href="#cb22-9"></a>                    <span class="fl">0.0018906078235370906</span><span class="ot">,</span></span>
<span id="cb22-10"><a href="#cb22-10"></a>                    <span class="fl">0.0021662522019363765</span><span class="ot">,</span></span>
<span id="cb22-11"><a href="#cb22-11"></a>                    <span class="fl">0.0017604492311402568</span><span class="ot">,</span></span>
<span id="cb22-12"><a href="#cb22-12"></a>                    <span class="fl">0.0015629781043435644</span><span class="ot">,</span></span>
<span id="cb22-13"><a href="#cb22-13"></a>                    <span class="fl">0.0019880688096594623</span><span class="ot">,</span></span>
<span id="cb22-14"><a href="#cb22-14"></a>                    <span class="fl">0.002500925711759432</span><span class="ot">,</span></span>
<span id="cb22-15"><a href="#cb22-15"></a>                    <span class="fl">0.0019362337711289173</span><span class="ot">,</span></span>
<span id="cb22-16"><a href="#cb22-16"></a>                    <span class="fl">0.0016825192087278592</span><span class="ot">,</span></span>
<span id="cb22-17"><a href="#cb22-17"></a>                <span class="ot">],</span></span>
<span id="cb22-18"><a href="#cb22-18"></a>                <span class="ot">[</span></span>
<span id="cb22-19"><a href="#cb22-19"></a>                    <span class="dv">132</span><span class="ot">,</span></span>
<span id="cb22-20"><a href="#cb22-20"></a>                    <span class="dv">150</span><span class="ot">,</span></span>
<span id="cb22-21"><a href="#cb22-21"></a>                    <span class="dv">120</span><span class="ot">,</span></span>
<span id="cb22-22"><a href="#cb22-22"></a>                    <span class="dv">165</span><span class="ot">,</span></span>
<span id="cb22-23"><a href="#cb22-23"></a>                    <span class="dv">23</span><span class="ot">,</span></span>
<span id="cb22-24"><a href="#cb22-24"></a>                    <span class="dv">88</span><span class="ot">,</span></span>
<span id="cb22-25"><a href="#cb22-25"></a>                    <span class="dv">230</span><span class="ot">,</span></span>
<span id="cb22-26"><a href="#cb22-26"></a>                    <span class="dv">129</span><span class="ot">,</span></span>
<span id="cb22-27"><a href="#cb22-27"></a>                <span class="ot">]</span></span>
<span id="cb22-28"><a href="#cb22-28"></a>            <span class="ot">]</span><span class="fu">,</span></span>
<span id="cb22-29"><a href="#cb22-29"></a>        <span class="fu">}</span></span>
<span id="cb22-30"><a href="#cb22-30"></a>    <span class="fu">},</span></span>
<span id="cb22-31"><a href="#cb22-31"></a><span class="fu">}</span></span></code></pre></div>
<p>Qmn example</p>
<div class="sourceCode" id="cb23"><pre class="sourceCode json"><code class="sourceCode json"><span id="cb23-1"><a href="#cb23-1"></a><span class="fu">{</span></span>
<span id="cb23-2"><a href="#cb23-2"></a>    <span class="dt">&quot;version&quot;</span><span class="fu">:</span> <span class="st">&quot;2.0&quot;</span><span class="fu">,</span></span>
<span id="cb23-3"><a href="#cb23-3"></a>    <span class="dt">&quot;Input&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb23-4"><a href="#cb23-4"></a>        <span class="dt">&quot;format&quot;</span><span class="fu">:</span> <span class="fu">{</span></span>
<span id="cb23-5"><a href="#cb23-5"></a>            <span class="dt">&quot;type&quot;</span><span class="fu">:</span> <span class="st">&quot;S8&quot;</span><span class="fu">,</span></span>
<span id="cb23-6"><a href="#cb23-6"></a>            <span class="dt">&quot;class&quot;</span><span class="fu">:</span> <span class="st">&quot;FXP&quot;</span><span class="fu">,</span></span>
<span id="cb23-7"><a href="#cb23-7"></a>            <span class="dt">&quot;params&quot;</span><span class="fu">:</span> <span class="ot">[</span> <span class="dv">0</span><span class="ot">,</span> <span class="dv">7</span><span class="ot">]</span></span>
<span id="cb23-8"><a href="#cb23-8"></a>        <span class="fu">}</span></span>
<span id="cb23-9"><a href="#cb23-9"></a>    <span class="fu">},</span></span>
<span id="cb23-10"><a href="#cb23-10"></a><span class="fu">}</span></span></code></pre></div>
</section>
<section id="ref_quant_mnist" class="level2">
<h2>Quantize a MNIST model</h2>
<p>Inside the <strong>X-CUBE-AI</strong> pack, a typical examples of quantization configuration file and associated user Python scripts are provided</p>
<pre><code>%X-CUBE-AI-DIR%/scripts/quantization/</code></pre>
<p>This example is a ready-to-use example or <strong>reference code</strong> to use the <em>GenericInputBatchGenerator()</em> class which is an alternative working for any type of input tensor formats (part of the <code>test_set_generation_mnist.py</code> file).</p>
<pre class="dosbatch"><code>    %X-CUBE-AI-DIR%\scripts\quantization
                                |- config_file_mnist.json
                                |- checkpoints
                                |    \_ mnist_fused.h5
                                \_ Mnist
                                    |_ test_set_generation_mnist.py
                                    \_ quantizer_algos_user.py</code></pre>
<div class="sourceCode" id="cb26"><pre class="sourceCode json"><code class="sourceCode json"><span id="cb26-1"><a href="#cb26-1"></a><span class="fu">{</span></span>
<span id="cb26-2"><a href="#cb26-2"></a>        <span class="dt">&quot;model_name&quot;</span><span class="fu">:</span> <span class="st">&quot;Mnist&quot;</span><span class="fu">,</span></span>
<span id="cb26-3"><a href="#cb26-3"></a>        <span class="dt">&quot;path_to_floatingpoint_h5&quot;</span><span class="fu">:</span> <span class="st">&quot;.</span><span class="ch">\\</span><span class="st">checkpoints</span><span class="ch">\\</span><span class="st">mnist_fused.h5&quot;</span><span class="fu">,</span></span>
<span id="cb26-4"><a href="#cb26-4"></a>        <span class="dt">&quot;quant_test_set_dir&quot;</span><span class="fu">:</span><span class="st">&quot;&quot;</span><span class="fu">,</span></span>
<span id="cb26-5"><a href="#cb26-5"></a>        <span class="dt">&quot;quant_test_ratio&quot;</span><span class="fu">:</span> <span class="st">&quot;0.3&quot;</span><span class="fu">,</span></span>
<span id="cb26-6"><a href="#cb26-6"></a>        <span class="dt">&quot;evaluation_test_set_dir&quot;</span><span class="fu">:</span><span class="st">&quot;&quot;</span><span class="fu">,</span></span>
<span id="cb26-7"><a href="#cb26-7"></a>        <span class="dt">&quot;batch_size&quot;</span><span class="fu">:</span> <span class="st">&quot;128&quot;</span><span class="fu">,</span></span>
<span id="cb26-8"><a href="#cb26-8"></a>        <span class="dt">&quot;modules_directory&quot;</span><span class="fu">:</span> <span class="st">&quot;.</span><span class="ch">\\</span><span class="st">Mnist</span><span class="ch">\\</span><span class="st">&quot;</span><span class="fu">,</span></span>
<span id="cb26-9"><a href="#cb26-9"></a>        <span class="dt">&quot;filename_test_set_generation&quot;</span><span class="fu">:</span> <span class="st">&quot;test_set_generation_mnist.py&quot;</span><span class="fu">,</span></span>
<span id="cb26-10"><a href="#cb26-10"></a>        <span class="dt">&quot;filename_quantizer_algos&quot;</span><span class="fu">:</span> <span class="st">&quot;quantizer_algos_user.py&quot;</span><span class="fu">,</span></span>
<span id="cb26-11"><a href="#cb26-11"></a>        <span class="dt">&quot;algorithm&quot;</span><span class="fu">:</span> <span class="st">&quot;MinMax&quot;</span><span class="fu">,</span></span>
<span id="cb26-12"><a href="#cb26-12"></a>        <span class="dt">&quot;arithmetic&quot;</span><span class="fu">:</span> <span class="st">&quot;Integer&quot;</span><span class="fu">,</span></span>
<span id="cb26-13"><a href="#cb26-13"></a>        <span class="dt">&quot;weights_integer_scheme&quot;</span><span class="fu">:</span> <span class="st">&quot;UnsignedAsymmetric&quot;</span><span class="fu">,</span></span>
<span id="cb26-14"><a href="#cb26-14"></a>        <span class="dt">&quot;activations_integer_scheme&quot;</span><span class="fu">:</span> <span class="st">&quot;UnsignedAsymmetric&quot;</span><span class="fu">,</span></span>
<span id="cb26-15"><a href="#cb26-15"></a>        <span class="dt">&quot;output_directory&quot;</span><span class="fu">:</span> <span class="st">&quot;Quantization_results&quot;</span></span>
<span id="cb26-16"><a href="#cb26-16"></a>        <span class="st">&quot;per_channel&quot;</span><span class="er">:</span> <span class="st">&quot;False&quot;</span><span class="fu">,</span></span>
<span id="cb26-17"><a href="#cb26-17"></a><span class="fu">}</span></span></code></pre></div>
<p>To use directly this file, original pre-trained Keras model (<code>cnn.h5</code> file) must be downloaded and copied in the <code>checkpoints/</code> directory (<a href="https://github.com/EN10/KerasMNIST/raw/master/cnn.h5">https://github.com/EN10/KerasMNIST/raw/master/cnn.h5</a>) with the <code>mnist_fused.h5</code> name. The associated data set is automatically downloaded and cached in the <code>~/.keras/datasets/</code> directory thanks the Keras <em>mnist.load_data()</em> function (see <em>create_test_generator()</em> function).</p>
<p>From the <code>%X-CUBE-AI-DIR%\scripts\quantization</code> directory, following command is used to launch quantization process:</p>
<pre class="dosbatch"><code>$ stm32ai quantize -q config_file_mnist.json</code></pre>
<p>The <em>create_test_generator()</em> function from the <code>test_set_generation_mnist.py</code> module calls a local <em>load_mnist()</em> function to download the public data set. Training samples are not used. “quant_test_ratio” parameter is used to create the expected <strong>quantization test-set</strong> and <strong>evaluation test-set</strong>.</p>
<p>Following code illustrates the minimal modifications to load the data set from the local <em>“quant_test_set_dir”</em> directory. Limited part of the original test set is used (randomly selected).</p>
<div class="sourceCode" id="cb28"><pre class="sourceCode python"><code class="sourceCode python"><span id="cb28-1"><a href="#cb28-1"></a>...</span>
<span id="cb28-2"><a href="#cb28-2"></a><span class="kw">def</span> create_test_generator(quant_test_set_dir, evaluation_test_set_dir,</span>
<span id="cb28-3"><a href="#cb28-3"></a>                          quant_test_ratio, batch_size):</span>
<span id="cb28-4"><a href="#cb28-4"></a>    ...</span>
<span id="cb28-5"><a href="#cb28-5"></a>    x_testset,y_testset <span class="op">=</span> load_mnist(quant_test_set_dir)</span>
<span id="cb28-6"><a href="#cb28-6"></a>    ...</span>
<span id="cb28-7"><a href="#cb28-7"></a></span>
<span id="cb28-8"><a href="#cb28-8"></a><span class="kw">def</span> load_mnist(test_dir):</span>
<span id="cb28-9"><a href="#cb28-9"></a>    <span class="im">import</span> numpy <span class="im">as</span> np</span>
<span id="cb28-10"><a href="#cb28-10"></a>    <span class="im">import</span> sys</span>
<span id="cb28-11"><a href="#cb28-11"></a>    <span class="im">import</span> os</span>
<span id="cb28-12"><a href="#cb28-12"></a></span>
<span id="cb28-13"><a href="#cb28-13"></a>    MNIST_SHAPE <span class="op">=</span> (<span class="dv">28</span>, <span class="dv">28</span>, <span class="dv">1</span>)</span>
<span id="cb28-14"><a href="#cb28-14"></a>    N_CLASSES <span class="op">=</span> <span class="dv">10</span></span>
<span id="cb28-15"><a href="#cb28-15"></a></span>
<span id="cb28-16"><a href="#cb28-16"></a>    <span class="bu">print</span>(<span class="st">&#39;Keras version  :&#39;</span>, keras.__version__)</span>
<span id="cb28-17"><a href="#cb28-17"></a>    <span class="bu">print</span>(<span class="st">&#39;Python version :&#39;</span>, sys.version)</span>
<span id="cb28-18"><a href="#cb28-18"></a></span>
<span id="cb28-19"><a href="#cb28-19"></a>    fdata_set <span class="op">=</span> os.path.join(test_dir,<span class="st">&#39;mnist.npz&#39;</span>)</span>
<span id="cb28-20"><a href="#cb28-20"></a>    <span class="cf">if</span> <span class="kw">not</span> os.path.isfile(fdata_set):</span>
<span id="cb28-21"><a href="#cb28-21"></a>      <span class="bu">print</span>(<span class="st">&#39;Upload the nmist data set with Keras service...&#39;</span>)</span>
<span id="cb28-22"><a href="#cb28-22"></a>      mnist <span class="op">=</span> keras.datasets.mnist</span>
<span id="cb28-23"><a href="#cb28-23"></a>      _, (x_test, y_test) <span class="op">=</span> mnist.load_data()</span>
<span id="cb28-24"><a href="#cb28-24"></a>    <span class="cf">else</span>:</span>
<span id="cb28-25"><a href="#cb28-25"></a>      <span class="bu">print</span>(<span class="st">&#39;Use the data set </span><span class="sc">{}</span><span class="st">&#39;</span>.<span class="bu">format</span>(fdata_set))</span>
<span id="cb28-26"><a href="#cb28-26"></a>      arrays <span class="op">=</span> np.load(os.path.join(fdata_set))</span>
<span id="cb28-27"><a href="#cb28-27"></a>      x_test, y_test <span class="op">=</span> arrays[<span class="st">&#39;x_test&#39;</span>], arrays[<span class="st">&#39;y_test&#39;</span>]</span>
<span id="cb28-28"><a href="#cb28-28"></a></span>
<span id="cb28-29"><a href="#cb28-29"></a>    msize <span class="op">=</span> <span class="bu">min</span>(<span class="dv">1000</span>, <span class="bu">len</span>(x_test))</span>
<span id="cb28-30"><a href="#cb28-30"></a>    np.random.seed(<span class="dv">2</span>)  <span class="co"># deterministic results</span></span>
<span id="cb28-31"><a href="#cb28-31"></a>    rchoice <span class="op">=</span> np.random.choice(<span class="bu">len</span>(x_test), size<span class="op">=</span>msize, replace<span class="op">=</span><span class="va">False</span>)</span>
<span id="cb28-32"><a href="#cb28-32"></a></span>
<span id="cb28-33"><a href="#cb28-33"></a>    x_test, y_test <span class="op">=</span> x_test[rchoice], y_test[rchoice]</span>
<span id="cb28-34"><a href="#cb28-34"></a></span>
<span id="cb28-35"><a href="#cb28-35"></a>    x_test <span class="op">=</span> x_test.reshape((<span class="op">-</span><span class="dv">1</span>, ) <span class="op">+</span> MNIST_SHAPE).astype(<span class="st">&#39;float32&#39;</span>) <span class="op">/</span> <span class="dv">255</span></span>
<span id="cb28-36"><a href="#cb28-36"></a>    y_test <span class="op">=</span> to_categorical(y_test, N_CLASSES)</span>
<span id="cb28-37"><a href="#cb28-37"></a></span>
<span id="cb28-38"><a href="#cb28-38"></a>    <span class="bu">print</span>(<span class="st">&#39;x_test&#39;</span>, x_test.shape)</span>
<span id="cb28-39"><a href="#cb28-39"></a>    <span class="bu">print</span>(<span class="st">&#39;y_test&#39;</span>, y_test.shape)</span>
<span id="cb28-40"><a href="#cb28-40"></a></span>
<span id="cb28-41"><a href="#cb28-41"></a>    <span class="cf">return</span> x_test, y_test</span></code></pre></div>
<section id="ref_quant_fd_mobilenet" class="level3">
<h3>FD MobileNet model example</h3>
<p>This second example illustrates the usage of the Keras API. <em>ImageDataGenerator()</em> class is used to generate batches of tensor image data with real-time data augmentation. Please refer to Keras documentation for more details. This API is typically used for the 4-D input tensor (batches, H, W, C) and for large data-set (see <code>test_set_generation_fdmobilenet.py</code> file).</p>
<pre class="dosbatch"><code>    %X-CUBE-AI-DIR%\scripts\quantization
                                |- config_fd_mobilenet.json
                                |- checkpoints
                                |    \_ fd_mobilenet_food_18_0.25_0.73526.h5
                                \_ Mnist
                                    |_ test_set_generation_fdmobilenet.py
                                    \_ quantizer_algos_user.py</code></pre>
<div class="sourceCode" id="cb30"><pre class="sourceCode json"><code class="sourceCode json"><span id="cb30-1"><a href="#cb30-1"></a><span class="fu">{</span></span>
<span id="cb30-2"><a href="#cb30-2"></a>        <span class="dt">&quot;model_name&quot;</span><span class="fu">:</span> <span class="st">&quot;fd_mobilenet_food_18_025&quot;</span><span class="fu">,</span></span>
<span id="cb30-3"><a href="#cb30-3"></a>        <span class="dt">&quot;path_to_floatingpoint_h5&quot;</span><span class="fu">:</span> <span class="st">&quot;.</span><span class="ch">\\</span><span class="st">checkpoints</span><span class="ch">\\</span><span class="st">fd_mobilenet_food_18_0.25_0.73526.h5&quot;</span><span class="fu">,</span></span>
<span id="cb30-4"><a href="#cb30-4"></a>        <span class="dt">&quot;quant_test_set_dir&quot;</span><span class="fu">:</span><span class="st">&quot;C:</span><span class="ch">\\</span><span class="st">train_Foodnet&quot;</span><span class="fu">,</span></span>
<span id="cb30-5"><a href="#cb30-5"></a>        <span class="dt">&quot;quant_test_ratio&quot;</span><span class="fu">:</span> <span class="st">&quot;0.9999&quot;</span><span class="fu">,</span></span>
<span id="cb30-6"><a href="#cb30-6"></a>        <span class="dt">&quot;evaluation_test_set_dir&quot;</span><span class="fu">:</span><span class="st">&quot;C:</span><span class="ch">\\</span><span class="st">val_Foodnet&quot;</span><span class="fu">,</span></span>
<span id="cb30-7"><a href="#cb30-7"></a>        <span class="dt">&quot;batch_size&quot;</span><span class="fu">:</span> <span class="st">&quot;128&quot;</span><span class="fu">,</span></span>
<span id="cb30-8"><a href="#cb30-8"></a>        <span class="dt">&quot;modules_directory&quot;</span><span class="fu">:</span> <span class="st">&quot;.</span><span class="ch">\\</span><span class="st">FdMobilenet</span><span class="ch">\\</span><span class="st">&quot;</span><span class="fu">,</span></span>
<span id="cb30-9"><a href="#cb30-9"></a>        <span class="dt">&quot;filename_test_set_generation&quot;</span><span class="fu">:</span> <span class="st">&quot;test_set_generation_fdmobilenet.py&quot;</span><span class="fu">,</span></span>
<span id="cb30-10"><a href="#cb30-10"></a>        <span class="dt">&quot;filename_quantizer_algos&quot;</span><span class="fu">:</span> <span class="st">&quot;quantizer_algos_user.py&quot;</span><span class="fu">,</span></span>
<span id="cb30-11"><a href="#cb30-11"></a>        <span class="dt">&quot;algorithm&quot;</span><span class="fu">:</span> <span class="st">&quot;MinMax&quot;</span><span class="fu">,</span></span>
<span id="cb30-12"><a href="#cb30-12"></a>        <span class="dt">&quot;arithmetic&quot;</span><span class="fu">:</span> <span class="st">&quot;Qmn&quot;</span><span class="fu">,</span></span>
<span id="cb30-13"><a href="#cb30-13"></a>        <span class="dt">&quot;weights_integer_scheme&quot;</span><span class="fu">:</span> <span class="st">&quot;SignedSymmetric&quot;</span><span class="fu">,</span></span>
<span id="cb30-14"><a href="#cb30-14"></a>        <span class="dt">&quot;activations_integer_scheme&quot;</span><span class="fu">:</span> <span class="st">&quot;SignedAsymmetric&quot;</span><span class="fu">,</span></span>
<span id="cb30-15"><a href="#cb30-15"></a>        <span class="dt">&quot;output_directory&quot;</span><span class="fu">:</span> <span class="st">&quot;.</span><span class="ch">\\</span><span class="st">Quantization_results&quot;</span><span class="fu">,</span></span>
<span id="cb30-16"><a href="#cb30-16"></a>        <span class="dt">&quot;per_channel&quot;</span><span class="fu">:</span> <span class="st">&quot;True&quot;</span><span class="fu">,</span></span>
<span id="cb30-17"><a href="#cb30-17"></a><span class="fu">}</span></span></code></pre></div>
</section>
</section>
</section>
<section id="references" class="level1">
<h1>References</h1>
<table style="width:92%;">
<colgroup>
<col style="width: 13%"></col>
<col style="width: 77%"></col>
</colgroup>
<tbody>
<tr class="odd">
<td style="text-align: left;">[1]</td>
<td style="text-align: left;">X-CUBE-AI - <em>AI expansion pack for STM32CubeMX</em><br />
<a href="https://www.st.com/en/embedded-software/x-cube-ai.html">https://www.st.com/en/embedded-software/x-cube-ai.html</a></td>
</tr>
<tr class="even">
<td style="text-align: left;">[2]</td>
<td style="text-align: left;">User manual - Getting started with X-CUBE-AI Expansion Package for Artificial Intelligence (AI) <a href="https://www.st.com/resource/en/user_manual/dm00570145.pdf">(pdf)</a></td>
</tr>
<tr class="odd">
<td style="text-align: left;">[3]</td>
<td style="text-align: left;">stm32ai - Command Line Interface <a href="command_line_interface.html">(link)</a></td>
</tr>
<tr class="even">
<td style="text-align: left;">[4]</td>
<td style="text-align: left;">Supported Deep Learning toolboxes and layers <a href="layer-support.html">(link)</a></td>
</tr>
<tr class="odd">
<td style="text-align: left;">[5]</td>
<td style="text-align: left;">Embedded inference client API <a href="embedded_client_api.html">(link)</a></td>
</tr>
<tr class="even">
<td style="text-align: left;">[6]</td>
<td style="text-align: left;">Evaluation report and metrics <a href="evaluation_metrics.html">(link)</a></td>
</tr>
<tr class="odd">
<td style="text-align: left;">[7]</td>
<td style="text-align: left;">FAQs <a href="faqs.html">(link)</a></td>
</tr>
<tr class="even">
<td style="text-align: left;">[8]</td>
<td style="text-align: left;">Quantization and quantize command <a href="quantization.html">(link)</a></td>
</tr>
<tr class="odd">
<td style="text-align: left;">[9]</td>
<td style="text-align: left;">Relocatable binary network support <a href="relocatable.html">(link)</a></td>
</tr>
</tbody>
</table>
</section>
<section id="revision-history" class="level1">
<h1>Revision history</h1>
<table>
<colgroup>
<col style="width: 32%"></col>
<col style="width: 24%"></col>
<col style="width: 44%"></col>
</colgroup>
<thead>
<tr class="header">
<th style="text-align: left;">Date</th>
<th style="text-align: left;">version</th>
<th style="text-align: left;">changes</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td style="text-align: left;"><strong>2020-05-12</strong></td>
<td style="text-align: left;">r2.0</td>
<td style="text-align: left;">initial version for X-CUBE_AI 5.1 based on quantize command section from the previous CLI article, adding overview</td>
</tr>
<tr class="even">
<td style="text-align: left;"><strong>2020-09-15</strong></td>
<td style="text-align: left;">r2.1</td>
<td style="text-align: left;">X-CUBE-AI 5.2 update, TFLiteConverter snipped code update</td>
</tr>
</tbody>
</table>
</section>



<section class="st_footer">

<h1> <br> </h1>

<p style="font-family:verdana; text-align:left;">
 Embedded Documentation 

	- <b> Quantization and quantize command </b>
			<br> X-CUBE-AI Expansion Package
				<br> r2.1
		 - AI PLATFORM r5.2.0
			 (Embedded Inference Client API 1.1.0) 
			 - Command Line Interface r1.4.0 
		
	
</p>

<img src="" title="ST logo" align="right" height="100" />

<div class="stnotice">
Information in this document is provided solely in connection with ST products.
The contents of this document are subject to change without prior notice.
<br>
© Copyright STMicroelectronics 2020. All rights reserved. <a href="http://www.st.com">www.st.com</a>
</div>

<hr size="1" />
</section>


</article>
</body>

</html>
